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AUTOMATED PRESENTATON OF ENTERTAINMENT 
CONTENT IN RESPONSE TO RECEIVED AMBIENT 

AUDIO 



TECHNICAL FIELD 

[0001 ] The inventions generally relate to presentation of entertainment content in response 
to received ambient audio. 

BACKGROUND 

[0002] With the advent of Napster and other peer-to-peer applications, the illegal 

distribution of audio files has reached epidemic proportions in the last several years. 
One way to combat this problem is the ability to acoustically analyze an audible wave 
pattern and generate a unique small "fingerprint" or "thumbprint" for that audio sample. 
The audio sample may then be compared to a huge database of fingerprints for all 
known music recordings. Such a database already exists in efforts to combat music 
piracy. 

[0003] One product that has been advertised to identify an unknown audio sample is by 
Audible Magic Corporation, 985 University Avenue, Suite 35, Los Gatos, CA 95032. 
Audible Magic Corporation advertises on their web site content-based identification 
software that can be integrated into other applications or devices. The software can 
scan a file or listen to an audio stream, derive fingerprints that will be used to identify 
the audio, and create an XML package that may be sent to ID servers via HTTP. A 
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reference database maintained by Audible Magic is used to provide positive 
identification information with a high level of data integrity using fingerprint information. 
[0004] Another product that has been advertised to identify an audio sample is an AudiolD 
System (Automatic Identification/Fingerprinting of Audio) by Fraunhofer Institut of 
Integrated Circuits IIS. The AudiolD System is described on the Fraunhofer web site as 
performing an automatic identification/recognition of audio data based on a database of 
registered works and delivering the required information (that is, title or name of the 
artist) in real-time. It is suggested that the AudiolD recognition system could pick up 
sound from a microphone and deliver relevant information associated with the sound. 
Identification relies on a published, open feature format to allow potential users to easily 
produce descriptive data for audio works of interest (for example, descriptions of newly 
released songs). 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0005] The inventions will be understood more fully from the detailed description given 
below and from the accompanying drawings of some embodiments of the inventions 
which, however, should not be taken to limit the inventions to the specific embodiments 
described, but are for explanation and understanding only. 

[0006] FIG 1 is a block diagram representation illustrating a system according to some 
embodiments of the inventions. 

[0007] FIG 2 is a block diagram representation of a flow chart according to some 
embodiments of the inventions. 
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DETAILED DESCRIPTION 
[0008] Some embodiments of the inventions relate to presentation of entertainment content 

in response to received ambient audio. 
[0009] In some embodiments, an apparatus includes an acoustic analyzer to identify 

received ambient audio and a content parser to select entertainment content associated 

with the identified audio for presentation of the entertainment content to a user. 
[0010] In some embodiments, a system includes an acoustic analyzer to identify received 

ambient audio, a content parser to select entertainment content associated with the 

identified audio, and a presentation device to present the selected entertainment 

content to a user. 

[0011] In some embodiments an ambient audio signal is received, the received ambient 
audio signal is identified, and entertainment content associated with the identified 
ambient audio is selected for presentation to a user. 

[0012] FIG 1 illustrates a system 100 according to some embodiments. System 100 
includes a microphone 102, an acoustic analyzer 104, an acoustic database 106, a 
content parser 108, a content database 110, and one or more presentation devices, 
including a television 1 12, a monitor 1 14 and a PDA (Personal Digital Assistant) 116. 

[0013] Microphone 1 02 automatically detects ambient audio (real time streaming audio). 
Acoustic analyzer 104 recognizes the ambient audio by consulting an acoustic database 
106. This may be accomplished, for example, by fingerprinting the ambient audio and 
consulting the acoustic database 106 for a match with that audio fingerprint. Such 
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fingerprinting techniques have been included, for example, in products of Audible Magic 
Corporation (Content-based identification API product) and Fraunhofer Institue of 
Integrated Circuits IIS (Automatic Identification/Fingerprinting of Audio) (AudiolD 
System). 

[0014] Audible Magic Corporation's content-based identification software may be used to 
scan a file or listen to an audio stream, derive fingerprints that will be used to identify 
the audio, and create an XML package that may be sent to a database that is used to 
provide positive identification information with a high level of data integrity using 
fingerprint information. 

[0015] Fraunhofer's AudiolD System performs an automatic identification/recognition of 
audio data based on a database of registered works and delivers the required 
information (that is, title or name of the artist) in real-time. Identification relies on a 
published, open feature format. 

[0016] Once the acoustic analyzer has identified the ambient audio (for example, a song) 
received by the microphone 102 the content parser 108 accesses content database 110 
to identify all entertainment content in that database that is associated with the identified 
audio. The content parser 108 can select for presentation all the identified 
entertainment content, randomly select for presentation some of the identified 
entertainment content and/or select for presentation some of the identified 
entertainment content based on certain selection criteria (for example, a selection by a 
user or a pre-selection of a certain type of content by a user, or selection or pre- 
selection of a certain type of content for certain audio or types of audio, time of day, day 
of the week, types of presentation devices currently available for use, and/or other 
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options). One or more presentation devices are coupled to the content parser for 
presentation of the entertainment content to a user (in some embodiments at the same 
time as the user is listening to the ambient audio). FIG 1 illustrates three types of 
presentation devices: television 112, monitor 114 and personal digital assistant 116. 
However, any combination of presentation devices (and arrangements with more than 
one of one type of presentation device) may be used. Examples of types of 
presentation devices that may be used according to some embodiments include any of 
the following or combination of the following: display, television, monitor, LCD, a small 
LCD (for example, a small LCD that is part of a stereo, hi-fi system, or car radio), 
computer, laptop, handheld device, cell phone, personal digital assistant, robot, 
automated toy, and audio speakers. Examples of types of entertainment content that 
may be presented according to some embodiments includes any of the following or 
combination of the following: pictorial, graphical, video, audio, audio-visual, textual, 
HTML, straight text, a textual document, straight text from the Internet (for example, 
from the Worldwide Web), and multimedia. Examples of entertainment content that 
may be presented according to some embodiments includes any of the following or 
combination of the following: music video, pictures, graphics, images, text, multimedia, 
a virtual DJ, a musical score, a moving toy, a stuffed animal, a moving robot, a 
computer desktop and a computer screensaver. 
[0017] In some embodiments acoustic analyzer 1 04 and content parser 1 08 may be 

included in a single device illustrate by a dotted line in FIG 1 (for example, a computer 
implemented in hardware and/or software). In some embodiments acoustic analyzer 
104 and content parser 108 may each be implemented in either hardware, firmware, 
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software and/or some combination thereof. In some embodiments such a computer 
may be a local computer local to the microphone 102 and the presentation devices 112, 
1 14 and/or 116. In some embodiments such a computer may be a remote computer 
remote from the microphone 102 and the presentation devices 1 12, 1 14 and/or 116. In 
some embodiments the acoustic database 106 may be local to the acoustic analyzer 
104, and in some embodiments the acoustic database 106 may be remote from the 
acoustic analyzer 104 (for example, coupled via a network connection, or accessible via 
the internet). In some embodiments the content database 110 may be local to the 
content parser 108, and in some embodiments the content database 110 may be 
remote from the content parser 108 (for example, coupled via a network connection, or 
accessible via the internet). In some embodiments the microphone 102 may be coupled 
to the rest of the system wirelessly. In some embodiments the presentation device (for 
example, television 112, monitor 1 14 and/or PDA 116) may be coupled to the rest of the 
system wirelessly. 

[0018] In some embodiments a system such as system 100 can automatically listen to 

ambient audio, recognize it, and then provide associated entertainment for presentation 
to a user. In some embodiments the entertainment content is directly related to the 
ambient audio (for example, music) being played in a given area (for example, the 
song's music video). In some embodiments, while listening to a CD (compact disc) a 
user could turn on a television set, display and/or monitor on which a music video 
corresponding to the song being played (or video, pictures, or related data of a musical 
group playing the song, for example). In some embodiments, a web page may be 
opened on a computer that relates to ambient audio being played (for example, the 
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musical group's web page, fan club web page or other web pages about the song 
and/or musical group). In some embodiments, for example, a user might come home 
and turn on a classical radio station playing a song such as Bach Aria. The screen 
saver of a user's computer suddenly begins showing pictures of Salzburg and/or other 
related Bach images, opens a web search (for example, using Google on Bach, 
Salzburg and/or Bach Aria), and/or shows a graphical musical score of the music being 
played (either accurate or merely generic to convey a musical mood). In some 
embodiments a child comes home, puts in his favorite CD, and his computer connected 
toy (for example, a robot or stuffed animal connected with a wire or wirelessly) begins to 
sing along with the song and/or dance to beat of the song. In some embodiments, 
alternative presentations can be provided. For example, additional drum beats are 
added to the song over some speakers, and/or additional drum beats are presented on 
a display, monitor, TV, etc. that gives the appearance that the computer, monitor, 
display, TV and/or other presentation device or attached peripheral are "jamming" with 
the beat. 

[0019] In some embodiments the identification of the received ambient audio may be 
performed locally to the ambient audio, remote from the ambient audio, and/or some 
combination thereof. In some embodiments the selection of the content associated with 
the identified audio may be performed locally to the ambient audio, remote from the 
ambient audio, and/or some combination thereof. In some embodiments, the 
presentation of the content to a user may be performed locally to the ambient audio, 
remote from the ambient audio, and/or some combination thereof. In some 
embodiments, a listener listens to the ambient audio and receives a presentation of the 
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content simultaneously. In some embodiments the presentation of the content is 
synchronized with the ambient audio (for example, the fingerprint of the audio includes a 
time stamp which may be used to synchronize the content presentation with the 
ambient audio). 

[0020] FIG 2 illustrates a flow chart diagram 200 according to some embodiments. Ambient 
audio is received at 202. The received audio is identified at 204 (for example, using an 
acoustic analyzer 104 and/or an acoustic database 106 as illustrated in FIG 1). The 
identified audio is used to select entertainment content associated with the audio at 206 
(for example, using a content parser 108 and/or a content database 100 as illustrated in 
FIG 1). The selected entertainment content is presented to a user at 208. In some 
embodiments the actual presentation at 208 is optional. 

[0021] Although some embodiments have been described in reference to particular 

implementations such as using particular types of acoustic analyzers and/or content 
parsers and/or requiring remote or local databases for comparison, other 
implementations are possible according to some embodiments. Further, although some 
embodiments have been illustrated and discussed in which entertainment content is 
selected for presentation and/or presented to a user, in some embodiments any content 
is selected for presentation and/or presented to a user. In some embodiments 
informational content is selected and/or presented to a user (for example, a museum 
displaying information about a particular song or piece of music, composer, singer, 
writer, etc.) 

[0022] In each system shown in a figure, the elements in some cases may each have a 
same reference number or a different reference number to suggest that the elements 
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represented could be different and/or similar. However, an element may be flexible 
enough to have different implementations and work with some or all of the systems 
shown or described herein. The various elements shown in the figures may be the 
same or different. Which one is referred to as a first element and which is called a 
second element is arbitrary. 

[0023] An embodiment is an implementation or example of the inventions. Reference in the 
specification to "an embodiment," "one embodiment," "some embodiments," or "other 
embodiments" means that a particular feature, structure, or characteristic described in 
connection with the embodiments is included in at least some embodiments, but not 
necessarily all embodiments, of the inventions. The various appearances "an 
embodiment," "one embodiment," or "some embodiments" are not necessarily all 
referring to the same embodiments. 

[0024] If the specification states a component, feature, structure, or characteristic "may", 
"might", "can" or "could" be included, for example, that particular component, feature, 
structure, or characteristic is not required to be included. If the specification or claim 
refers to "a" or "an" element, that does not mean there is only one of the element. If the 
specification or claims refer to "an additional" element, that does not preclude there 
being more than one of the additional element. 

[0025] Although flow diagrams and/or state diagrams may have been used herein to 
describe embodiments, the inventions are not limited to those diagrams or to 
corresponding descriptions herein. For example, flow need not move through each 
illustrated box or state, or in exactly the same order as illustrated and described herein. 
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[0026] The inventions are not restricted to the particular details listed herein. Indeed, those 
skilled in the art having the benefit of this disclosure will appreciate that many other 
variations from the foregoing description and drawings may be made within the scope of 
the present inventions. Accordingly, it is the following claims including any amendments 
thereto that define the scope of the inventions. 
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